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Abstract 

Timeout control is a simple mechanism used when 
direct feedback is either impossible, unreliable, or too 
costly, as is often the case in distributed systems. Its 
effectiveness is determined by a timeout threshold pa- 
rameter and our goal is to quantify the effect of this 
parameter on the system behavior. In this paper, we 
extend previous results to the case where there are N 
transmitting nodes making use of a common commu- 
nication link bandwidth. After deriving the stochastic 
hybrid model for this problem, we apply Infinitesimal 
Perturbation Analysis to find the derivative estimates of 
aggregate average goodput of the system. We also de- 
rive the derivative estimate of the goodput of a transmit- 
ter with respect to its own timeout threshold which can 
be used for local and hence, distributed optimization. 



1. Introduction 

Timeout control is a simple mechanism used in 
many systems where direct feedback is either impos- 
sible, unreliable, or too costly. This is often the case 
in distributed systems, where remote components can- 
not be observed by a controller (or several distributed 
controllers) and information is provided over a network 
with unreliable links and delays. A timeout event is 
scheduled using a timer which expires after some time- 
out threshold parameter. This defines an expected time 
by which some other event should occur or a "grace pe- 
riod" over which some response about the system state 
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is required. If no information arrives within this pe- 
riod, a "timeout event" occurs and incurs certain reac- 
tions which are an integral part of the controller. In dis- 
tributed systems, where usually control decisions must 
be made with limited information from remote com- 
ponents, timeouts provide a key mechanism through 
which a controller can infer valuable information about 
the unobservable system states. In fact, as pointed out 
in [28 1, timeouts are indispensable tools in building up 
reliable distributed systems. 

This simple reactive control policy based on time- 
outs has been used for stabilizing systems ranging 
from manufacturing to communication systems fl4l . 
Dynamic Power Management (DPM) ll2ll,lfT3l.ll22l and 
software systems 0,(9] among others. Despite its wide 
usage, quantifications of its effect on system behavior 
have not yet received the attention they deserve. In 
fact, timeout controllers are usually designed based on 
heuristics which may lead to poor results; an impor- 
tant example can be found in communication protocols, 
especially TCP (see [D,G3),GD,EU and references 
therein). There is limited work on finding optimal time- 
out thresholds. For instance, in lEUl . a single queueing 
model is used for an Automatic Guided Vehicle (AGV) 
system and shared tester equipment. In DPM where the 
aim is to minimize the average power consumption, IPT31 
and J2] propose a timeout control scheme with the aid 
of the theory of competitive analysis. Also, in Q, a 
Markov process model is used and Infinitesimal Pertur- 
bation Analysis (IPA) [4],| 10 1 is applied to calculate the 
optimal timeout threshold values. In communication 
systems, |[TT1 . lfT8ll , lfT9l have attempted to find the op- 
timal TCP retransmission timeouts by making assump- 
tions on the probability of the transmission failure in the 
system. Finally, in [12J optimal web session timeouts 
are calculated so as to reduce the probability of falsely 
ending a web session in time sensitive web pages. All 



such approaches are limited by their reliance on the dis- 
tributional information about the stochastic processes 
involved. 

In this paper, we consider the timeout control in the 
context of Discrete Event Systems (DES), so that the 
controller output includes a response to either a time- 
out event or an event carrying information about net- 
work congestion. However, since stochastic DES mod- 
els can be very complicated to analyze, we rely on re- 
cent advances which abstract a DES into a Stochas- 
tic Hybrid System (SHS) and, in particular, the class 
of Stochastic Flow Models (SFMs). A SFM treats the 
event rates as stochastic processes of arbitrary general- 
ity except for mild technical conditions. The emphasis 
in using SFMs is not in deriving approximations of per- 
formance measures of the underlying DES, but rather 
studying sample paths from which one can derive struc- 
tural properties which are robust with respect to the ab- 
straction made. This is the case, for instance, with many 
performance gradient estimates which can be obtained 
through IPA techniques for general SHS ||6l, l|25l . In ad- 
dition, a fundamental property of IPA in SFMs (as in 
DES) is that the derivative estimates obtained are in- 
dependent of the probability laws of the stochastic rate 
processes and require minimal information from the ob- 
served sample path. This approach has proved useful in 
optimizing various performance metrics in serial net- 
works [23 1, systems with feedback control mechanisms 
11271 . scheduling problems [15|,[16|, and some multi- 
class models [5|,|24|. 

In ATI , we set forth a line of research aimed at 
quantifying how timeout threshold parameters affect 
the system state and ultimately its behavior and perfor- 
mance. We adopted a communication system model 
consisting of one transmitter submitting packets to a 
network with stochastic processing times. We showed 
how a SFM of a timeout-controlled distributed system 
can be obtained and then used to optimize the timeout 
threshold. This paper extends the results in HTl to mul- 
tiple communication links and hence, extends this prob- 
lem to broader multi-class problems with communica- 
tion delays. Like ifTTl . we directly control the timeout 
parameters for network communication performance. 
Additionally, moving from a singe class problem in ifTTll 
to multiple classes, we show how by carefully selecting 
the processing rate of each class, the shared communi- 
cation channel operates according to a First Come First 
Serve (FCFS) policy. We also discuss how we can eas- 
ily extend the analysis to non-FCFS frameworks. Fi- 
nally, we also extend previous work on timeout control 
by removing any dependence on distributional informa- 
tion. 

This paper is organized as follows: In Section|2] we 



define the SFM of the system. In Section [3] we apply 
IPA techniques to average goodput at each transmitter, 
as well as total goodput of the system aggregating the 
rates for all the transmitting nodes. We conclude with 
Section|4] 



2. Stochastic Flow Model (SFM) 

We are interested in systems where N transmitting 
nodes send data to their destinations through a shared 
channel. Associated with each node n = 1 ... ,N, is a 
"class" n of data packets or tasks which are to be pro- 
cessed in the channel and reach their destinations in a 
timely manner. Like the case of a single communica- 
tion link, after each transmission by node n, a timely 
acknowledgement (ACK) is expected from the receiv- 
ing end. For transmitter node n to operate normally, 
the Round-Trip Transportation (RTT) time calculated 
for each node n should be less than a timeout threshold 
parameter 9„ > 0. If no ACK is received within 9„ units 
of time from the transmission, a timeout occurs causing 
node n to retransmit the timed out data (previously sent 
at t — 9 n ) to the same destination. The control param- 
eter vector is defined as 9 = [81, 62, . ■ ■ , ftv] • We con- 
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Figure 1 . The SFM for the timeout system 



sider a SFM for this problem where N upstream nodes 
are competing to get their content through a shared net- 
work channel. The channel latency represents the RTT 
delay in the underlying DES. The SFM for this system 
is shown in Fig. Q]and is observed over a finite period 
of time [0,T]. Associated with this system are several 
nonnegative stochastic processes all defined on a com- 
mon probability space (D.,^,P). The transmitters are 
shown as upstream nodes receiving exogenous inflow 

processes {X„(t)}, n = 1, ,JV and copy/timeout flow 

processes {y n (t.9)} intended for retransmission. The 
process {z„(t,9)} models the amount of pending data 



to be transmitted by node n and evolves according to 

dz n (t,6) 



Zn(t,0) 



if Zn (t,9) = Q,y n (t,e)<<l> n (t,9) 

y n {t,6) - (j)„(t,0) otherwise . 

(1) 



where 



\j/ n (t,e) = ?i n (t)+y„(t,e) 



(2) 



and {(j)„(t , 9)} is the maximal transmission rate process 
at node n. Thus, the actual transmission rate from node 
n to the network buffer is defined as 



0*0,0) = 



Wn(t.,e) ifz n (t,e) = o,\ir n (t,e)<<i> n (t,e) 

<p n (t,9) otherwise 

(3) 



In this paper, we assume that the transmitters have 
an infinite supply property as follows: 
Assumption 1 . z„ (t , 9 ) > for all t G [0 , T] . 

By 0), this implies On(t, 9) = § n (t,9) > for all 
t G [0,T] , so (j) n (t,9) can henceforth be replaced by 
a„(t,9) for the rest of the discussion. 

We model the shared network channel with a buffer 
where the transmitted fluid from node n accumulates 
and takes a share x„(t , 9) > of the total buffer content 
x(t , 9) = Y.n x n(t, 9). The processing rate of the fluid is 
governed by nonnegative service processes {/3„(f , 9)}, 
n=\,...,N. 

Furthermore, for each n, we define the independent 
process {B„(t)} as the maximal processing rate of class 
n when no other class is being processed by the network 
resource. Since, j3„(f,0) is the processing rate in the 
presence of other fluid classes competing for a share 
of processing power in the network, we naturally have 
P„(t,9)<B n (t). 

At any time t G [0,T], x n (t, 9) in the network fol- 
lows the dynamics 



x n (t,9) 



dx n (t, 9) 
dt+ 



if x(t, 9) =0,a(t, 9) <j3(t, 9) 

a„(t , 9) - /3„ (f , 9) otherwise 

(4) 



where a{t,9) = L„a„(f,0) and j3(r,0) = Y,„Pn(t,9). 
We also define the process 



Pn(T,9)dT = X n (t-W,9) } , 



w n (t, 9) = min 

' (5) 

as the waiting time process of class n G {l,...,N}. 
We assume that the value of the processes /3„(f , 9) and 



x n (t — w, 9) are known over t G [— w,0] as the initial 
pieces of information required to calculate w(t,9) at 
t = 0. Definition (0 has a very close relation to the 
"time-to-empty" process as defined in [26|,[8 1 since one 
way to interpret (f5]) is as the earliest time it takes the 
server to process the content at t — w. 

Using the waiting time process, we can define the 
timed-out (hence, worthless) portion of class-n fluid in 
the network buffer for which w„(t , 9) > 9„. This por- 
tion needs to be retransmitted by the upstream node n. 
Under normal conditions, the network generates an ac- 
knowledgement flow indicating a successful transmis- 
sion previously made at t — w n (t, 9). In case of a time- 
out at node n - i.e. the moment w„(t, 9) exceeds 9„ - a 
copy of the fluid sent at time t — 9 n is generated and will 
be put back in node n for retransmission. This is shown 
as the feedback flow process y„(f , 9) > in Fig. [T] To 
summarize, for all t G [0, T], we have 



Yn{t,9) 







On (t — 9 n ,9) otherwise 



ifw„{t,9)<9„ 



(6) 



The upstream node treats this copy flow with higher pri- 
ority and tries to retransmit it as soon as possible. This 
is inline with many retransmission-based communica- 
tion protocols. Looking at Fig[T] we should explicitly 
show the flow y„(f , 9) creating contents ahead of those 
created by X n (t). However, we refrained from this to 
avoid complicating the illustration. 

We look at average goodput as the communication 
performance metric defined as 



J(9,T)=E[G(9,T)} 



G{9J)=Y j 



a n {t,9)-2Yn{t,9) 



dt 



(7) 
(8) 



where, like the one-dimensional problem 1171 . we use 
2y„(f,0) to not only penalize the retransmissions but 
also to account for the worthless timed out fluid in the 
network buffer which should be processed by the net- 
work resource anyway. 

Before proceeding, we make the following assump- 
tion on the system which we also made in ifTTll : 
Assumption 2. The network buffer is lossless. 

This assumption is merely for simplifying the ex- 
position and can be removed at the expense of having 
new state variables tracking the loss volume of each 
class. 

2.1. FCFS Implementation 

The definition of the waiting time (0 is contingent 
upon having a FCFS policy implemented in the network 
buffer. One of the differences between a single trans- 
mitter model in ifTTI is in how to ensure this policy is 



preserved. We achieve this by carefully selecting the 
network processes. 

In general, depending on the processes j3„(f , 9), if 
two fluid particles of class n and m, n ^ m are transmit- 
ted at times X\ and Xz, ii > Ti, if j3 m (f , 9) is sufficiently 
larger than fi„(t, 9), their order of leaving the network 
may be reversed. We can implement both FCFS and 
non-FCFS policies by carefully defining the processing 
rates /3„(f , 9), n — 1, . . . ,N. We limit the details on the 
non-FCFS policies to Remark Q] where it is shown that 
the extension of the results to the non-FCFS policies is 
straightforward. 

Recall that /3„(f , 9) < B,,(t) since some part of the 
resource is consumed by other classes. Moreover, just 
as in the DES where a slow service for one type of cus- 
tomer increases the waiting time of the other customers 
behind it, if B„(t) is small, it not only means slow pro- 
cessing for class n, but also a decreasing effect on other 
processing rates j3,„(f , 9), m ^ n. The availability rate 
of fluid class n at the network server can be defined as 

& n (t,9) = a n (t-w n (t,9),9) We [0,7-]. (9) 

Accordingly, at any time t S [0, T], the utilization of the 
service associated with class n — I, ... ,N is as follows: 



pn{t,6) 



B n {t) 



(10) 



Hence, at each time t , p n (t, 9) is a measure of how en- 
gaged the server is with the fluid of class n. The follow- 
ing theorem ensures all these characteristics are deliv- 
ered by carefully defining j5 n (t,9),n — l,...,N. Com- 
pared to [ 26 1 , 1 8 1 , it allows for different maximal service 
processes for each class. 

Theorem 1. Let w 6 M + . Assuming the initial 
conditions w„(0, 9) = w for all n, if 



On{t,9) 



the following statements are true: 



«»M) y . _ rn Tl 



(ii) 



(i) There exists a common waiting time process 
w(t, 9) such that w n (t, 9) = w(t, 9) for all n and 

all? e [o,r]. 

(ii) If < ti < tz, then t\ - w(t\,8) <t% — w(t 2 , 9) 

Proof: When x(t , 9 ) = 0, the buffer clearly operates 
according to FCFS, so we only consider the case where 
x(t, 9) > 0. By proving statement (/), we show the fluid 
particles at the head of the queue must have arrived at 
the same time; Statement (ii) shows that regardless of 
the class, fluid particles leave the system with the order 



they have been transmitted. 

We start by proving statement (i). Differentiating the 
term in brackets in (0 with respect to t reveals 

P„(t, 9) -[I- w n (t, 0)]p n (t - w n (t, 0), 9) = 

[i-w„(t,e)]x n (t-w n (t,e),e) 



which after regrouping the terms yields 

Pn(?,8) 



W„(t,d) = l- 



cc n (t-w n (t,9),9) 



= 1 - 



P„(t,9) 

&n(t,ey 



(12) 



Using ( fTTT > in the last equation gives 
1 



w n (t,0) = 1 - 



HmPm(t,0)' 



Vte[0,T). (13) 



for which we consider the constraint w„(t) > 0. No- 
tice that the RHS is independent of n. Hence, with the 
initial condition that w„(Q, 9) = w for all n, W n (t, 9) = 
w m (t, 9) for any pair n,m, m and the FCFS opera- 
tion when w„(t, 9) is defined for all n. Finally, we can 
define the waiting time dynamics of a fluid differential 
at the head of the network buffer as 



Mt,6) 



l 



EmPm(*,< 



if x(t,e) = 

otherwise 



(14) 



This proves part (i) in the theorem statement. 

For part (ii) notice that by definition, a fluid particle 
which is at the head of the queue at time t has arrived at 
t — w(t, 6). Therefore, we only need to show that for any 
fi,*2 G [0, T] such that t\ < tz, we have t\ — wi(?i, 9) < 
h — wz(t 2 , 9). We can write t\ —w(t\ , 9) < tz~w(t 2 , 9). 
It follows that 



h-t 2 < w(h,9)-w(tz,9) 

= w(t\ , 9) - w(t\ ,9)+ I w(t, 9)dx 

= - ( 2 w(x,8)dx 

r- 1 

= h-t 2 - I ; —dX 



1 



E m Pm(T,0) 

which, by ( fT4l i. gives 
0< 

Notice that r -K fl , > 0. This proves part (ii) and the 

Liu Pm{V,0) r 

theorem. BWith a more 

complicated proof, we can show the theorem statements 



L„Pm(T,0) 



dx. 



are true even if Assumption[TJdoes not apply. However, 
we do not consider it in this paper. 
Remark 1: We can come up with schemes other than 
FCFS and generalize the SFM analysis. For example, if 
by defining 

SCn(t,B) 



Vn. 



L, n 6lm(f,0)' 

we share the resource capacity according to relative 
availability of the fluid classes at the server, ( TT2T > be- 
comes 



w„(t) = 1 - 



B„(t) 



Vre[0,r]. (15) 



Now, the right-hand side of this equation is n- 
dependent, which breaks the FCFS rule. 

For the rest of this paper, we make the following 
assumption: 

Assumption 3. B n (t) = B(t) for all t e [0,T] and 
n=\,...,N. 

We can remove this assumption at the expense of 
more complexity which diverts the focus from the main 
purpose of this analysis. Assumption [3] is not limit- 
ing in the present problem as the communication chan- 
nels treat the packets from different sources or classes 
equally. Moreover, a byproduct of Assumption[3]is that 



j3M) = J>M) = £ 



&n(t,9) 



m B m (r) 



(16) 



which means the total processing rate of the channel is 
independent of the timeout rates chosen. 

2.2. Transmission Control 

We say node n is operating in a normal period 
when w(t,8) < 9 n . In this mode, we assume that 
the inflow rates are determined according to the pol- 
icy 7C\ n which is designed to increase the transmission 
rate a„(t,9). We adopt a second policy 7t2. n which 
applies when the node is in the timeout period, i.e., 
w(t,9) > 9 n . Policy 7T2n is reactive and aims at re- 
ducing the transmission rate until the network chan- 
nel comes out of the congestion and can satisfies the 
requirement w(t, 9) < 9 n again. There can be many 
choices for the policies %\ n and 7C2 n . F° r me sa k e of 
analysis, we choose the policies as follows: 

£2& £ * .(17) 

cc n (t, 9) = 0£„, min if w(t, 9)>9„. 



2.3. Stochastic Hybrid Model 

Viewed as a SHS, we can conceive of the following 
SFM operation modes throughout the sample path: We 
refer to a period over which x(t,9) > and x(t, 9) = 0, 
as a Non-Empty Period (NEP) and Empty Period (EP), 
respectively. Moreover, we denote the periods over 
which w(t , 9) > 9„ and w(t , 9) < 9„ by TOP,, and NP„, 
respectively. 

Let Tfc, k = l,...,K, be the SFM event times ob- 
served in a sample path of the system over the interval 
[Q,T]. We also define To = and Tk+i = T for nota- 
tional convenience and let e^ be the event occurring at 
Tfc. We are interested in the following set of events: 



g = {E x ,E Bl [x > 0], [x =0], [w > 9 n ],[w < On], 



(18) 



Here, E x and £g respectively refer to random jump 
events in any XJf), n = 1,...,N and B(t) which are 
not affected by 9. We call the events with this prop- 
erty exogenous. The start and end of a NEP is marked 
by the events [x > 0] and [x = 0], respectively, [w > 
9„] and [w < 9„] are the events defining the start and 
end of a TOP,,. Since these events are dependent on 
the system states, we categorize them as endogenous 
events. When a timeout [w > 9„] event occurs, accord- 
ing to (fTTI ). 0C„(t, 9) drops. This discontinuity will then 
be reflected in a n (t,9) and consequently, by ( fTTI ). in 
P„(t, 9). We refer to the event of a„(f + , 9) ^ a„(t~, 9) 
by [ct£ a^]. As the last system event, [y* ^ Yn) i s an 
event of discontinuity in 7„(f,0) strictly inside a TOP. 
Since the start and stop of a TOP,, is already defined 
by [w > n ] and [w < n ], by ©, [7+ ^ 77] only re- 
flects a discontinuity in a„ (t — 9„,9) when w(t,9) > 9„. 
This directly affects the objective function ((H) as it is 
related to y„(t , 9) defined by ©. We call [a+ ^ a^} 
and [7+ ^ Yn] induced events because these events are 
bound to occur after the occurrence of a triggering event 
in the past. Generally, the triggering event can be either 
an exogenous, endogenous or itself an induced event. 
More information on the induced events can be found 
inEl. 

The delays between [w > 9„] and its associated 
[a+ 7^ 6c~] and [7+ y~] call for new state variables. 
The role of these state variables is to provide timers 
which trigger when [w > 9„] occurs and measure the 
amount of time until the associated induced event. For 



each event time T# we define the state variables 

Ai(*.'.0)=( otherwise Vf ^< 

(19a) 

hl (k,T+9)=l -< tk > 6) if u e ^.[ w>0 «] (1%) 
v * ' otherwise 



; , r. i 1 if 3n : et = [w > 9„] , , ^ 

Wfc.fjS) = 4 „ u • Vf>T jt , 

' otherwise _ ' 



v ' * ' ' otherwise 



(20a) 
(20b) 



with the constraint hi(k,t,6) >0,i= l,2,k= l,...,K. 
To identify the active timers on [t£,%|_i), for each k = 
l,...,K,we define the index set 

<$>k = {m<k: 3i s.t. hi(m,t, 9) > V t e [x k , X k+ \)}. 

(21) 

3. Performance Optimization by IPA 

The following assumption ensures existence of the 
IPA derivatives: 

Assumption 4. With probability 1, no two events can 
occur at the same time unless one causes the other. 

Considering the objective ©, let us define the in- 
dex set 

a n = {k-.w(t,e) > e„ v* e [x k ,x k+l )} (22) 

marking all the event times in TOP,, including its start. 
The other possible event in Q.,, is [a+ ^ a~] which by 
Assumption[4] cannot occur independently of [w > G„] 
and [w < 8„]. Since by ©, when w(t, 9) > 9„, we have 
Y„(t, 9) — a„(t — 9 n , 9), we can decompose ([8]) as fol- 
lows: 



G(T,9) = Y,G n (T,9), 



(23a) 



G n (T,9)=\ / a„{t,9)dt 



k=0 Jr k 



2 E / * +1 On(t-d„,d)dt\, n = l,... t N. 
ken,, Jt >! > 



3.1. IPA Estimation 



(23b) 



Let us define x' k ■ = for k = 0, . . . ,K + 1 and 
j = l,...,N. Also, for a real valued function f„(t,9) 



associated with node n, we define the partial derivatives 
f nJ {t,8) = df f e ,6) ■ Differentiating G n (T,d) with re- 
spect to 9j and noticing by To = 0, and Tr+\ — T, that 
for any j reveals 



T o,./ — T k+ij 



dG„(T,9) 



K 

k=\ 



" 2 E i+l.j a "([^k + l ~ On] ) - </«„([^ - GJ + ) 

/tefi„ - 

/"**+' , , , , da n {t-9 n , , 
52/ a„j(t)dt+ I - 



k=0 jT k 



d9: 



(24) 

The expression in the last integral in d24i > can be written 



as 



da n (t - 9„,9) _ 9a n (x, 9) 



d9 



j 

da„(r,t 



d6j 



d{t-9 n ) 
, 96, 



a n {t-9 n ,9) 



where d e \z=t-e„ = K,]^-^ e )- Moreover, ac- 
cording to ([T7j, a„(t - 9„, 9) = r n if w„(t - 6„,6) < 9 n 
(i.e., node n not in a TOP,, at t — 9 n ). Noting that 
d^g 8 "' 1 = — 1 only if j = n and 0, otherwise, we find 



da„(t-9„,9) 
d9i 



--a' n Jt-9 n ,9) 



r„ ifw(t-6 n ,6) < 6 n ,j = n 
otherwise 

(25) 

Looking at the performance objective as well as the 
conditions under which an event is triggered, the state 
vector of the system is comprised of a„(t, 9), J n {t : 9), 
n = l,...,N, x(t,9) and w(t,9). However, note that 
J n {t,9) = CCn(t - 9 n ,9) when w„(t,9) > 9„. Thus, its 
derivative can be obtained from that of a„(t,9). Fur- 
thermore, Assumption [3] allows us to have the follow- 
ing useful lemma which reduces the number of states to 
only two: 

Lemma 1. If the network channel operates ac- 
cording to the FCFS policy, we get 

9w(t,9) 1 9x(t,9) 



d9j a(t,0) d9j 



Vte[0,T),j = l,...,N, 



where x(t, 9) = x(t - w(t , 6) , 6) for all t e [Q,T]. 

Proof: Differentiating © with respect to 9j j G 
{1, ...,#} gives 

dp n (T,9) 



v' nJ (t,d)p n {t-w n (t,e),e)+ f 



d9j 



-dx 



dx n (x,9) 



d9i 



dx n (x,9) dx 



dx d9i 



T=f — M>„(f,8) 

= </? - w„(f , 0), 0) -x„(t - w„{t, 9), 9)w' n j (t, 9) 



where we have x n (t — w n (t,9),9) = a„(t — 
w n (t, 9), 9) - p n (t - w n (t, 9), 9). Thus, 



dP„{T,9) 

-w„(i,e) d9j 



dx 



w ' n j(t,e)p„(t-w n (t,9),e)+ f 

Jt-Wnif 

= x' nJ (t-w„(t,9),9) 
-[a n {t-w n (t,6),8)-p„(t-w n (t,e),e)}w' n j(t,9) 

Regrouping terms yields the following for all t G [0,T]: 



*:^-vv n M),e)-;/_ WnM) ^ 

a„(t,9) 



dx 



Using Theorem Q] and by Assumption [T] w„(t,9) = 
w(t,9) for all n = l,...,N and all t G [0,T). Since 
w' n (t,9) = w'(t,9) for all n 6 {1,...,N} we can add all 
the numerators and denominators and write 



dw(t,9) _ 



a»(r,0)? 



x' n>j (t-w(t,e),e) 

dp„(T,9) 



d9j 



dx 



Since 0) = a{t,9) and £„- 

= 0, we find that for j = 1, 



://3„(f.6 



#(0 



,iV, 



<9w(f,0) 



1 <?x(f-w(f,0),0) 



d9j a(t,9) d9j 



1 <?i(f,0) 
a(t,9) d9j 



3.2. IPA equations 

Before proceeding, we provide a brief review of the 
IPA framework for general stochastic hybrid systems 
as presented in @. If s(t,9) G R M is the state vec- 
tor of the SFM, IPA specifies how changes in 9 influ- 
ence s(t , 9) and the event times Tfc and, ultimately, how 
they influence interesting performance metrics which 
are generally expressed in terms of these variables. Let 
us assume that over an interval [x k , x k+ \ ), the SFM is at 
some mode during which the time-driven state satisfies 
* = fk{s,9,t) for some f k : 



Leu'(r)^e 
all state derivatives. It is shown in (6l that, for any 

t G [x k ,X k+ \), s'(t) satisfies: 

for t G [x k , x k+ \ ) with boundary condition: 

= Jit) + < [A-i (s) - /*«)] ( 27 ) 



[o,r)-> 



pAf v. iraiV 



be the Jacobian matrix for 



for k = 0, . . . ,K if is continuous at and otherwise, 

/(T+) = r'(x k ). (28) 

An exogenous event at x k is not a function of 9 
so t£ ■ = for any j = l,...,N. In our model these in- 
clude^ and £g. However, for every endogenous event, 
<?£ at x k there exists a continuously differentiable func- 
tion g k : R" X @ -> E such that = min{f > t k _\ : 
^ (s(t ; 0), 0) = 0}. It is shown in @ that 



n -i 



39 



ds S [Xk 



(29) 



if e k G S is endogenous and defined as long as 
~tfk{\) + 0. In addition to the exogenous and en- 
dogenous events, we also have induced events. Using 
the results in in this case, if e k is induced by <?,„, 
m < we can write 



dXk_ 

dOj 



[hi(m,x k ,9)]' 



ds(x. 



dht 



d6j 

dhj(m,x k - ,9) 

39; 



(30) 



where is the gradient vector of partial derivatives 
hi with respect to state variables and depending on 
whether [a+ ^ a~] or [y+ ^ y~\ has occurred, we use 
hi or /i2 as defined in ( fT9l ) and ( l20l >. 

3.2.1. Event-time derivatives. We will determine ^ 

for each event in $ and for each 9j, j — I, . . . ,N. We 

exclude Ex and £g as they are exogenous events with 

zero event-time derivative. Recalling a(t) = 

a(t) = I„«»( f ) and = LnPn(t), the following 

lemma gives the derivatives for exogenous and induced 

events: 

Lemma 2. Under policies 7ti n and 7^,,,, 

n = 1,... ,N, for any j = 1 , . . . ,N, we have 



(/) 

(in) 



lfe k 
lfe k 
lfe k 



^ = 



[x > 0]: 
[* = 0]: 



%j 



; [w > 9„] or [w < 9 n \. 
i B =jft(^.fl)— ^ 



(iv) Ife fc : 



a(T,-,e)-/3(T,-) 
[a+^a„-]: 



(v) 



x' mj a{x mi 9) 



If e k = [tf?Y n ]: = in,j + !«=j 
where 1„ =/ is an indicator function being 1 when n = j 
and 0, otherwise and x m < x k is the time of the 
triggering event for the induced event at x k . 



Proof: Starting with part (z), we can invoke 
with gk(t,9) = a(t,Q) -/3(f). Noticing ^ = 1 and 
% = -1, we find that %&(<tj-, 0) + fyfoxj-) = 

&(T-,0)-j8(T-). Moreover, - 



°^l k - . However, by (O we know that j3 (f ) = 5(f) and 
is independent of the control parameters. Thus, we get 

= da< de ' e ^ • I nsertm g me results in the expres- 
sion for d29l i verifies part (/). For part (//), notice that 
we have an endogenous event with gk(t,6) — x(t,9). 
Noticing that when x(t,9) > 0, x(t,9) = f x (t,6) = 
a(t,d)-P(t). Hence, d -§f x {\,0) = o(t^,0) - 

/^). Also, ^1 = These give f| = 



; jr. — which is exactly the claim of 

part (ii). For part (Hi), we have gk(Xk, 0) — w(x k , 9) - 
9„. By JTJI i, and Assumption[3] we have 



#(t,6)=f w (t,8) = l 



1 



1 



5(0 



Since §| = 1, we find ^f w (^,9) 

finally, — ^ — - — Wj Clearly, ^ - i„ =7 . 

By LemmaQ] we get 

dw(x k ,9) ^ 1 ^([T t -w(T t ,fl)]-,8) 
d9j a{x~,9) d9j 

Thus, we find that 



LmPm(t,9) a{t,9) 

= &(t,6)-P(f) 
&(t,0) 

dgk t ^ _ »{\ ) 



<90 ; - a{x k ,9) V dfy 
Putting these results in ( f29b gives 

^ _ 1 dg k {x k ,9) 

l„ =j a(x k , 9) - 



6(^,0) -J5(0 



which completes the proof for part (z'z'z). For case (zv), 
we have the condition hifa, 0) = which assuming the 
triggering time is T,„ can be written as 

f Tk 

h 1 (m,X k ,9)=x(x m ,9)- P(t)dt = 0. 



Notice that, h\{x k , 9) = — j8(T ft ) and 
dhi{m,x~,9) _ dx(x m ,9) dx m _ . 4 

h a07 p ^ 



50, 



<90, 



^-,0) , ^ A(T - )0) + ^ (Tm+) . 



50; 



<?0; 



50/ 



Since, at X m we have the event of discontinuity of 
On, and since by Assumption |H cannot coincide 
with this event, we have j3(r m ) = J3(t+). Using this 
in the above expression by knowing that f x {x tn ,9) = 
a(z-,9)-P(x-) reveals 



dhi(m,T k ,9) dx(x~,9) 



d9j 



99j 



< n ,./«(T,„,0)- 



Finally, notice that hi(m,x k ,9) is not directly depen- 
dent on the value of any state variable at x k , hence 
the first term in the paranthesized expression in (f30b 
is 0. Putting all the results into (f30l proves part (iv). 
Finally, for case (v), by (l20l i. we have /i2(m,f,0) = 
x(x m ,G) — (t — X m ) for t 6 [x m ,X k ] with the boundary 
condition h.2(m,X k) 9) = 0. Applying the same proce- 
dure proves part (v) and the whole lemma. ■ 

3.2.2. State Derivatives. Here, we find the derivative 
equations for the state variables x and a. We do not 
include other states in the IPA calculations as by d25b 
and Lemma [T] they can be calculated in terms of the 
derivatives of x and a: 

A) Analysis at event times: We start by finding the 
derivative update equation of a„(t,9). Notice that un- 
like ifTTl where O£„(f,0) was a continuous function of 
time, here, by definition a„(f,0) has a discontinuity 
whenever event [w n > 0„] occurs. In general, we have 
the following lemma: 

Lemma 3. Concerning state variable a n (t, 0), 



da„(x k + ,9) 
98, 



= 0, ife k = [w> 9 n }. (31) 



Also, for the rest of the events, we have 



da„{x+.,9) 

39; 



da„(x+,9) 
99 j 

l„ =j a(T k .e) ^j-L 

' a(T k ,6)-fi(T-) 



ife k = [w<9„] 
otherwise 

(32) 

Proof: Equation OTT ) is immediate from ( f28b not- 
ing r(t) — a„. m ; n is independent of 9. Focusing on the 
second part, the proof of this lemma is also straightfor- 
ward by using d27l ). Notice that the only event e k (ex- 
cluding [w > 9„]) at whose occurrence time the differ- 
ence fa n (\,9) -/c&,(t^,0) is non-zero is e k = [w < 



0„] where by O, we get fafc, 0) -fatf, 9) = 
— r„ — —r„. Using this information in < f27b proves the 
lemma. ■ 
Next, for any real-valued function /, we define 
Aa n (t k ) = On{\,6) - 04, (t/, 0) for k = 1 , . . . ,K. We 
derive the discrete update equations for state variable x. 
Lemma 4. Concerning state variable x(t, 0), we 

have 



+ < 



dx(r. 



Aa n {x k ,9) 




MM — -Jw— 

a(T,-.e)-j8(T.-) 



if <?£ = [x = 0] 

if ej c =[w> 9„] 
otherwise 

(33) 

Proof: The proof follows by straightforward appli- 
cation of d27l i to state variable x. When e k = [x = 0], we 
get /»(T t -,0) = a(T-,6) - j3(T fe -) and /*(<r+,0) = 0. 
Hence, by using ^ from Lemma|2] we find that 



Lf^-,0) -/,(%+ 



[a(T,-,0)-j8(Tr)] 



59, 



a(T t -,0)-/3(T; 



50; : 



which proves the first condition. When e k = [w > 0„], 
we again invoke Lemma [2] for this event and use it in 
(ED considering f x {x^, 9) - /*(t+, 0) = 0^,(^,0) - 
a(T^, 0). For other events, case by case analysis shows 
that fx(tr, 0) — fx{tti 9) is on ly non-zero whenEg oc- 
curs. However, this is an exogenous event with ^ = 
for all j = l,...,N. M 
B) Analysis between event times: We start from 
CCn(t,9). Recall that by ( ITTb and Assumption [T] 
/«„(?, 0) = f^{t,9) for all n and t G [0,T). Since r„ 
is not a function of 0, differentiating f an (t. 0) with re- 
spect to 0j reveals 



<?0; 



0, We [0,r) and j=l,...,N. (34) 



For x(r, 0), © implies that for all t G [0, T), 

fx(t,9) = 





a(t,0) 
therefore, for j = 1 , . 

dfxjt,8) _f 



■ ,N, 



ifx(f,0) = O,a(r,0) <j3(f) 
otherwise 



ifx(f,e) = O,a(f,0)<jB(f) 
otherwise 

(35) 



Using (|25T > and Lemma[T]and applying the IPA for- 
mulas given above completes the state derivative esti- 
mations. 

3.2.3. IPA Implementation. We assume that the IPA 
derivatives for the states and event times over the inter- 
val [t — w(t, 0),f ) are available for IPA calculations at 
t. 

Recall that we are interested in estimating the 
derivative of the average cost (0 by finding the deriva- 
tive of the sample function ([8]l which can be calculated 
from d24l . However, evaluating (1241 is contingent upon 
the knowledge of </f,0), f/a "^"- e > and z' kj . By 

d25l l. t/a "^/g 8 "' 9 ^ can be readily obtained given that the 
past values of a' n •(*, 0) and w(t — 0„, 0) are known. By 

Lemma|2] %[ f can be obtained if the derivatives 

dxjr.O) 



e dx(t,e 



and 



da(t.G) 

are available. The latter is the value 



dxjt.e 

at t — w(t,9). The derivative dx jf^ itself 



can be found by evaluating ( f33b at event times [x = 0] 
and [w > 0„], n — l,...,N and integrating d35l l between 
these events. Furthermore, ^fe^ can be calculated by 
evaluating ( f3Lb at the occurrences of [w > 0„] and ( T32t 
at those of [w < 0,,]. Notice that by ( [34-b , no integration 



of df ^: e) is needed. 



4. Conclusions 

We extended the results in 1 17 1 to multiple commu- 
nication links and laid down the conditions under which 
the network buffer works according to the FCFS policy. 
We also showed how extensions to non-FCFS policies 
are possible. Under the FCFS policy, we derived the 
derivative estimates of average goodput at each trans- 
mitter and used it to find the sensitivity of the aggregate 
goodput with respect to timeout thresholds. 
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